Remote media capture device activation

ABSTRACT

Remote video capture device activation enables a video capture device to be activated from a server or another client device. In various embodiments, a mobile client may receive an instruction from an entity remote to the mobile client. The mobile client may interpret the instruction into at least a command. The mobile client may retrieve a software event corresponding to the command according to a predefined command to event mapping. The mobile client may further execute an event handler that handles the software event responsive to the command. The event handler may initiate a media capture device of the mobile client to capture a data asset of a crime scene and may further associate the data asset with a case identifier for the crime scene.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This patent claims priority to U.S. Pat. No. 8,311,983, entitled“Correlated Media for Distributed Sources,” issued on Nov. 13, 2012, andU.S. Pat. No. 8,311,983 claims priority to U.S. Provisional PatentApplication No. 61/173,459, filed on Apr. 28, 2009, which are herebyincorporated herein by reference in their entirety.

TECHNICAL FIELD OF THE SUBJECT MATTER

This subject matter of the present application relates to aggregatingand correlating distributed media.

BACKGROUND OF THE SUBJECT MATTER

Media data comes in many forms including, but not limited to, video,audio, still images, and text. Presently, media data is captured, thatis recorded, and stored on a storage media that is dependent on the formof media data. For example, video is captured by video cameras, audio iscaptured via microphone and recorders, and still images are captured bycameras.

Currently, video cameras and digital recorders are used for a wide rangeof applications. While the use of video cameras and digital recorders istypically associated with personal events. There are many applicationsof the use such devices for commercial purposes including security andsurveillance. For example, police car video cameras are used to recordstop encounters.

As described above, more than one form of media may be used to capturean event. For example, a security camera and a digital audio recordermay capture both video and audio data respectively from a crime scene.Afterwards, a police officer or security supervisor may add textcaptions to the video using computer-based software or on-camerafunctions. Combining different forms of media for presentation is termedmultimedia, and accordingly there may be multimedia capture of securityevents and crime scenes.

Presently, captured media is most commonly stored as digital data,thereby becoming a data asset. Digital data assets may be streamed tousers or consuming devices in real time, or may be captured and latertransported as a file for streaming.

The consumer electronics revolution has made digital video cameras,digital still cameras, and digital recorders ubiquitous. Accordingly,commoditized video cameras and digital recorders have become availablefor security applications. Digitization and miniaturization has led tothe production of video cameras that can fit in a mobile phone with everimproving resolution. Further, the advent of commoditized compact memoryhas enabled large amounts of video data to be stored in such devices, ina cost effective manner. As of this writing, 16 gigabytes (GB) ofstorage space can store 40 hours of video data with average resolution.Accordingly, large amounts of digital data assets may be captured frommany different sources and in many different media. Furthermore, theindividuals that capture a security event or crime scene with a cameraor recorder need not necessarily be related. For example, at a crimescene, there may be surveillance cameras that were stationed in the arealong before the scene; there may be police officers with mobile camerasand recorders, and another police officer taking still shots with adigital camera.

With the Internet, digital data assets may be shared in both in editedand non-edited form. In the past, files were shared simply bytransferring peer-to-peer, such as e-mailing files or uploading to a LANbased server. Later, digital data assets were posted and distributed viaweb pages via internet protocols. Currently police officers and securitypersonnel can post and distribute digital data assets to a centralizedlocation via web services, with facilities to search and tag on postedassets. In this way, different recordings of the same crime scene mightbe aggregated to help solve a crime case, regardless of who originallycaptured or uploaded the digital asset.

In general, there is presently a critical mass of digital data assetsthat can be correlated and combined together. For example, panoramicsoftware can stitch together different still photos taken at the sametime of the same event and result into a single photo. Different videoand audio feeds may be mixed together to make a composite rendering.However, such efforts are typically manual in nature and use relativelyshort media clips.

At present, automating the correlation and combination of multimedia ofrelatively large long data assets, such as those hundreds or thousandsof hours in length is not done. Moreover, recording metadata to aid incorrelating the data assets with other data assets is not presentlydone. Finally, using such correlating metadata to automate correlationsof data assets into a combination presentation is not presently done.

SUMMARY OF THE SUBJECT MATTER

The embodiments described herein relate to a comprehensive system tostore data assets, associate correlating metadata, share data assets ineither a peer to peer manner or via a web service or equivalent,retrieve data assets, and present data assets either singularly or invarious combinations.

Method embodiments described herein may include associating anidentifier along with correlating metadata such as date/timestamp andlocation. The identifier may then used to associate data assets that arerelated to a particular incident. The identifier may be used as a groupidentifier on a web service or equivalent to promote sharing of relateddata assets. Additional metadata may be provided along with commentaryand annotations. The data assets may be further edited and postprocessed. The data assets may be then presented either singularly or invarious combinations, either locally or in a custom application. Acustom application may be hosted at a network operation center as wellwith capabilities of directing collection of data assets.

System embodiments described herein may include a mobile client that isbased on a modified cell phone. The mobile client may include videocameras and other media recording capabilities, location servicefunctionality such as global positioning system (GPS) functionality, andfull networking capabilities. The mobile client may also support apreview mode by which the video camera captures not only an incident atthe time of recording starting, but also several seconds of videopreceding the event as stored in a preview buffer. The mobile client mayfurther include a custom charger in the form of inductance coils and afile manager that provides file system capabilities specific to a mobiledevice. Even further, the mobile client may include a tamper proofchassis and optional features for physical secure storage.

The mobile client hardware may be exposed to application software via aneventing system that allows programmers to add event handlers for customsoftware events. The system as disclosed may also include end to endapplications covering use of external advertising and annotations andtools.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive examples are described with reference tothe following figures. In the figures, the left-most digit(s) of areference number identifies the FIG. 1 n which the reference numberfirst appears. The use of the same reference numbers in differentfigures indicates similar or identical items or features.

FIG. 1 is a diagrammatic illustration showing individual actors,hardware, and software in a web service embodiment of correlating mediafor distributed services.

FIG. 2 is a diagrammatic illustration showing individual actors,hardware, and software for sharing and correlating data assets inaccordance with a web service embodiment of correlating media fordistributed services.

FIG. 3 is a diagrammatic illustration showing individual actors,hardware, and software for capturing an event, sharing data assets, andcorrelating the data assets into a final result. In a peer-to-peerembodiment of correlating media for distributed services.

FIG. 4 is a flowchart of an exemplary method for correlating media fordistributed services.

FIG. 5 is a diagrammatic illustration showing key components of hardwarefor an exemplary mobile client in accordance with various embodimentsfor correlating media for distributed services.

FIG. 6 is a diagrammatic illustration of correlating media fordistributed services in a cloud computing environment.

FIG. 7 is a diagrammatic illustration showing key components of softwarefor an exemplary mobile client in accordance with various embodimentsfor correlating media for distributed services.

FIGS. 8A and 8B are diagrammatic illustrations of exemplary non-clientsoftware applications in accordance with various embodiments forcorrelating media for distributed services.

FIG. 9 is a diagrammatic illustration of the individual actors,hardware, and software applied to a personal use scenario of correlatingmedia for distributed services.

FIG. 10 is a diagrammatic illustration of the individual actors,hardware, and software applied to a law enforcement scenario ofcorrelating media for distributed services.

DETAILED DESCRIPTION OF THE EMBODIMENTS Combining and CorrelatingDigital Data Assets

The embodiments described herein pertain to methods, systems, andapparatuses for capturing digital media as data assets, associatingcorrelating metadata with those assets, and retrieving and correlatingdata assets at least partially based on the correlating metadata. Inthis way, the task of correlating disparate data assets into a coherentcombination may be automated.

There are many different types of combinations of data assets. Suchcombinations may be of similar media, as in making a panoramic compositephoto by stitching together different digital photos. Further, suchcombinations may be of different media, such as adding a custom audiotrack to a video. Such combinations may even be of the same context,such as providing different views of the same police, fire and/orsecurity incident. In extreme situations, different contexts may becombined such as with a mashup. Such combinations need not be compositesand may be presented as various different correlated media assetsdisplayed side by side, or as combined into an integrated whole.

To support automating such a varied range of combination presentations,the embodiments described herein allow a rich range of correlatingmetadata to be associated with a data asset. When two or more dataassets are to be combined, as a composite or otherwise, the metadata mayprovide a referent by which each data asset may be correlated. Commonmetadata may includes, but is not limited to, date/time stamp andlocation metadata.

For example, a video clip may have date/time information stored for eachframe, and an audio clip may have date/time information stored for eachtrack segment. The video clip and the audio clip may be combined into amultimedia presentation by correlating the date/time stamps of the twofiles. In this case the correlation is a synchronizing of the two filesand the presentation is a composite of the two files. Of course, thenumber of such files that may be so combined is not limited to just two.

Correlation can be based on multiple metadata values. For example,multiple still photos might be stored not only with date/time stampmetadata, but also with location metadata, possibly from a globalpositioning satellite (GPS) stamp. A software tool that collects allstored still photos taken within a window of time, for example during asecurity or police response to a crime incident, and close to the sceneof a crime, may combine the photos of the incident into a sequence ofpictures with which for investigation purposes. Here the correlation isboth by time and location, and the presentation is a non-compositesimultaneous display of different data assets.

Correlating metadata can be based on a set of custom fields. Forexample, a set of video clips may be tagged with an incident name.Consider three field police officers each in a different city and in adifferent time zone, recording videos and taking pictures at exactly atmidnight on New Year's Day 2013. Each officer might tag their videos andtheir still photos with “New Year's Day 2013 Security Watch”. A softwaretool may collect all stored videos with that tag, and may furtherprovide a presentation rotating between the videos and photos for theevent. Here the correlation is made using a custom tag, is not specificeither to time or location, and is a non-composite display of differentdata assets.

This degree of flexibility means that correlations can be both byabsolute referent and by relative referent. Most incidents occur at anabsolute time and location, and incidents may occur at differentlocations (such as a bank robbery occurring at a bank, and a breakingand entering at a power main to shut off the power to enable therobbery). Another example would be the security of an event overmultiple locations such as the election or Olympics which are held atdifferent venues in a city or different cities. In other situations,incidents, such as a high speed chase may span multiple counties. Theabove is a correlating example of synchronizing video and audio by meansof a relative referent, whereby the time stamp need not be of absolutetime, but could potentially be synchronized to the beginning of the dataclips. The above example of the New Year's Day event is an extremeexample of correlating by a relative referent. The “New Year's Day 2013Security” tag is arbitrary, and correlates data assets without regard totraditionally absolute referents of date/time and location. This exampleis also known as date/time shifting and location shifting.

Exemplary Platform for Correlating and Combining Data Assets

FIGS. 1 and 2 illustrate an exemplary platform for correlating andcombining data. FIG. 1 illustrates capture of data. FIG. 2 illustratespresentation of data.

In this exemplary platform, event 100 is to be captured by recordingdevices 111-114 each of which is to capture some part of event 100 asrepresented by scopes 101-104, respectively. The data captured by therespective scopes 101-104 may be uploaded in the form of files to a webservice 120 by users with accounts 121-123 that can transfer files fromtheir respective devices 111-114. The web service 120 groups users intogroups, e.g., users with accounts 121-124 belong to group 130. Note thatdevice 113 can be associated with multiple user accounts 122 and 123.Note that the user with account 124 does not have a device but can stillparticipate in a group. The users thus organized can store the dataassets into a central storage 150. The user with account 125 which doesnot belong to group 130 may potentially belong to any number of othergroups on the web service, or alternatively may join group 130. The userwith account 125 may have devices (not shown) and upload to the centralstorage 150. Users 141 and 144, with accounts 121 and 124 respectively,may log on later to tag, comment, annotate, or otherwise associatemetadata with the uploaded files. The following discussion describesthis configuration in more detail.

Event 100 may represent the capture of any set of media data. Usually anevent consists of a discrete real-world event such as a crime incident.However, some users may wish to combine different events in differentlocales and time zones. These different events may not even have anycontext in common, save for a user's preferences. For example, a policesupervisor may be looking for patterns of assaults in a jurisdiction.Each individual assault was perpetrated independently, but studying theaggregate data, the police supervisor may detect areas of high risk.

An event can be captured by one of many commonly available devices111-114. Exemplary devices include, but are not limited, to videocameras, digital audio recorders, or digital still cameras. Anotherexample of a recording device might be a laptop that stores GPS samplesor other location service samples to track location. Devices 111-114 mayrecord data in analog format but then convert the data to digital formatlater for sharing. Devices 111-114 may further have the capability ofcapturing different types of media at the same time. For example, avideo recorder usually captures video and audio at the same time.Alternatively, many digital cameras support both video and still modesof capture. Notwithstanding the many features of present day devices,usually a single device at best can only capture a subset of an event asrepresented by scopes 101-104. Each scope only covers a subset, but acombination of the scopes may provide a more complete capture of theevent.

Devices 111-114 may generally store captured media either as files orstream the captured media to a remote device that will persist the data.Eventually the captured media may be stored as a digital data asset,usually as a file. In the alternative, data may be persisted on analogtape.

Devices 111-114 may have the ability to associate metadata either beforecapture or at time of capture. For example the respective devices mayprovide for a user to provide a default tag for all files such as cameraidentifier (ID). During capture, the respective devices may storedata/time stamp information or location service tracking location datasuch as GPS with the file.

An example of a default tag associated before time of capture is anEvent ID (also called a Pounce™ ID) that may be used to indicate whichfiles are candidates for correlating for an event. Generally an Event IDcorresponds to a unique event. The end to end process is discussed inthe context of FIG. 4. Metadata association is discussed in more detailbelow in the context of FIG. 5.

In the exemplary platform in FIG. 1, each device 111-114 belongs to atleast one user (not shown), and each user has an account 121-123. In thecase where data is being shared peer to peer, accounts may not benecessary since devices 111-114 may access each other without the needfor intermediate storage and because the user is operating only onedevice. However, in the FIG. 1 exemplary platform, note that the userassociated with account 123 is operating both devices 113 and 114.Further note that devices 111-114 need not have been operated at thesame time. For example, a user might record video in a police car on adevice for an incident and on a separate device record video and/oraudio that may or may not begin or end at the same time.

Accounts 121-124 may be aggregated on web service as 120 as a group 130.As a default, a group may be identified to include all users with datafiles with the same Event ID. A group may also be either a predefined ora self selecting group, for example a set belonging to a securityagency, or a set of all police officers belonging to the homicidedivision, or even a set of officers seeking to share data regardless ifthey bellowing to an organized or unorganized group. A group may also berelated to some other grouping external from the web service, e.g., alaw enforcement Case ID seeking all data assets that might have capturedan event such as a crime in progress.

In this exemplary platform, users with accounts associated with devices111-114 then may upload files to central storage 150. Users may belongto zero, one, or more groups. An example of where a user belongs to zerogroups is includes a user having initially established an account,although an alternative embodiment may include a user creating a defaultgroup. An example of a user belonging to multiple groups includes avideo file belonging both to a group to capture a security event and toa group showing what other security personnel were doing on a given day.

Users with accounts in a group may also have the ability to add metadataafter capture of the media data. As illustrated by users 141 and 144,adding metadata may include, but is not limited to, commenting on filesby manually adding meta data or by automation, such as tagging, whichmay be performed via software tools and by scripting.

Users with accounts are not limited to associating data assets withmetadata. Users with accounts in a group also have the ability to editthe data assets directly. For example, a still photo may be uploaded andlater cropped using a digital photo editor. The editor may be providedas on-line tool as part of the web service or, alternatively, may beprovided as desktop software by which the user may download the file,edit, and re-upload the edited file.

User account 124 pertains to a user who is not associated with a device.In general, user accounts may be associated with zero, one, or moredevices. Because users can add metadata after capture, a user may stillparticipate in a group without being responsible for providing dataassets to the group. This is a common scenario for master editors aswell as mixers who use data assets as samples.

In this exemplary platform embodiment, the result is a central store 150of data assets that have been associated with correlating metadata frombefore, during, and after time of capture. At least some subset of themetadata may be used for correlation.

FIG. 2 includes references to features first introduced with referenceto FIG. 1 to continue discussion of this exemplary platform embodiment.Central store 150 may be accessed by external users for metadatainclusion and final presentation. An external user 144 may accesscorrelation and presentation tool 220 that translates user 144'soperation into a query performed by search module 210. At least some ofthe query criteria may be based on correlating metadata. The searchmodule 210 may retrieve data assets, from central store 150, that matchthe criteria of the query. Additionally the search module 210 mayretrieve external data sources. The search module 210 may forward querycriteria to an external correlation module 260, which retrieves dataassets from an external data store 230 and associates the necessarycorrelation metadata with the data assets. Examples of external datainclude, but are not limited to, map data and to advertising data.External data may be retrieved by the search module 210 along with dataassets from central store 150. The search module 210 may then return thecombined data to the correlation and presentation tool 220. Accordingly,additional correlations may be made, others correlations removed, andthe data assets may be presented side by side, in rotation, orpotentially in composite form to user 144. A final presentation may besaved as a single file or single distribution, and persisted on datastore 240 where it may then be viewed by other users 250, regardless ofaffiliation with a group or from the web service. The followingdiscussion describes this process in more detail.

The correlation and presentation tool 220 is represented as a singleentity. However, correlation and presentation may be implemented indifferent modules, for example as different dynamic link libraries(DLLs) within a single application or as separate plug-ins from abrowser. The correlation and presentation tool 220 may have a query tool(not shown) that provides for simple queries, e.g. retrieve all dataassets relating to “New Year's Day 2013 Security Watch.” Alternatively,the tool may provide the back end infrastructure for a domain specificapplication such as a law enforcement dispatch tool. Several suchapplications, including applications for law enforcement and fortransportation are discussed below.

The correlation and presentation tool 220 may invoke search module 210in the form of a query. The query may be a SQL query that retrieves dataassets according to a series of where clauses. For example in thefollowing SQL pseudo-code:

select data_asset_id from data_asset_table, group_table wheredata_asset_table.group_id = group_table.group_id and group_id =[group_id] and data_asset_table.date_time_stamp = [date_time_stamp]the correlation and presentation tool 220 may retrieve all data assetidentifiers in a specified group at a particular time. In thealternative, the query may simply be a parameterized DLL call invocationas in, for example, the following function declaration pseudo-code:

ReturnDataAssetByTimeStamp[(group_id], [date_time_stamp]).

In addition to filtering and correlating via predefined fields, theembodiments described herein may support filtering on custom metadata.One option to support custom metadata is to support a number of customfields within a relational database. In this way, a simple textcomparison may return all data assets when a custom tag equals, e.g.,“New Year's Day 2013 Security Watch.” In the alternative, somerelational databases may support stored procedures that can invokebinaries. For example, MICROSOFT SQL SERVER™ allows invocation of COMobject and ORACLE 10G™ allows invocation of JAVA™ objects.

In the above examples, the actual binary to execute the queries is thesearch module 210. The search module may not only invoke data assets inthe central store 150, but also zero, one, or more external databases230 via the external correlation module 260. Example external databasesmay store, as examples only, mapping data, advertising data assets,domain data such as public Securities and Exchange Commission (SEC)filing data, or integration with other applications such as lawenforcement case management databases. At this stage, correlations mayoccur in the form of join statements in SQL, or binary equivalents.

The external correlation module 260 may be responsible for returningdata assets from the external database 230 along with correlationmetadata. The data assets may be returned with at least an Event ID toallow general correlation with the other data assets returned from thecentral store 150. Any metadata that was captured before or duringrecording may be converted to a format correlatable with the data assetson the central store 150. For example, if a date-time stamp is in adifferent format, the external correlation module may convert thedate-time stamp to the central store 150 format.

The external correlation module 260 may have access to multipledatabases, so it may use one external database for data assets andanother database for metadata. One example is a Joint PhotographicExperts Group (“JPEG”) file that has GPS information which is correlatedwith a geolocation advertising database that links ads specific to theJPEG's location.

When the search module 210 retrieves data assets and associatedmetadata, it may return the data to the correlation and presentationtool 220 optimally as a rowset. Specifically, it may return a binaryenumerable set of rows in which each data row contains a data asset ID,an Event ID, related metadata and at least an accessible reference tothe data asset so that the correlation and presentation tool 220 mayactually retrieve the data asset. In the alternative, when a highbandwidth dedicated connection is available, the data assets may beserved as a binary large object (“BLOB”). In the alternative to arowset, the search module may return the data asset id, event id,metadata, and data asset reference as an eXtensible Markup Language(“XML”) file.

Once at the correlation and presentation tool 220, the data assets maybe further correlated for presentation. For example, once the dataassets are local to the correlation and presentation tool 220, furtherfiltering may be performed resulting in changing correlations. Forexample, when the rowset returns all data assets focusing on aparticular crime incident, it may be possible to retrieve only the dataassets focusing only on a particular person under surveillance. Becauseof the extent of possibilities, correlations may be driven by a customrules engine.

The correlation and presentation tool 220 may further support displayingthe data assets both simultaneously and as composite. One example ofsuch displaying would be to have multiple synchronized videos of asecurity incident playing on small screens and providing a control toview images from a selected one of the small screens on a large screen.Another example would be to stitch together individual still images intoa single composite. Yet another example would be to display a firstvideo followed immediately by a second correlated video.

Composite presentations may require editing capabilities on the subjectdata assets. For example in stitching, the different JPEGs taken fromdifferent cameras and different locations will vary in shading, colorfiltering, and the like. Software post processing modules may beresident in the correlation and presentation tool 220 to apply digitalphoto processing to correct and homogenize the JPEGs to be stitched.

As a result of the correlation and presentation functions beingintegrated into the same tool 220, the correlation engine itself may beused to determine which data assets are to be post processed. Forexample, the correlation function may identify which JPEGs have anaverage darkness beyond a predetermined threshold. In the alternative,the correlation engine might calculate an average darkness andhomogenize the images to that average.

After post processing, the correlation and presentation tool 220 may beused to present the final presentation to the user 144. The user mayapply other processing as desired.

The user 144 has the option of storing the final presentation as asingle multimedia file or, in the alternative, as a single distributionof multiple files. The single file or distribution may be stored on anintermediate data store 240 where other users 250 may view or access.

The intermediate data store 240 may be a networked share drive or a fileserver accessible via the web. In the alternative, it may be portablemedia such as a memory stick or DVD-ROM. Access to the data on theintermediate data store 240 may be encrypted or password protected.

In this exemplary platform embodiment, the result is a single multimediafile or single distribution that combines multiple data assets that havebeen correlated for consumption by a user.

Exemplary Peer to Peer Embodiment

FIG. 3 illustrates a peer to peer embodiment, whereas FIGS. 1 and 2illustrate a web service centric embodiment.

Event 310 may be captured by mobile clients 321 and 322, both configuredin accordance with the embodiments described herein. Mobile client 321may then capture a portion of event 310 via video recording 331. Mobileclient 322 may similarly capture a portion of event 310 via videorecording 332.

During the recording, both mobile clients 321 and 322 may store metadataidentifying the date/timestamp and location of the recording.Potentially each client may also store data assets with differentdate/timestamps and locations.

After the event is over, mobile client 321 and mobile client 322 mayattempt to establish a peer to peer, mesh, LAN, or WLAN connection overthe Infrared Data Association (“IRDA”) protocol. Specifically, if mobileclient 321 initiates, the mobile client may first generate an Event IDbased on the mobile client's device number. The user of the mobileclient may add additional metadata, e.g., the name of the event. Mobileclient 321 may then send this metadata over to mobile client 322 viaIRDA. When mobile client 322 sends an acknowledgement, a link 340 may beestablished. Henceforth, any data assets transferred over the link maybe tagged with the Event ID and associated metadata.

Because this is a peer to peer collection, mobile clients 321 and 322can enumerate each other's data assets. Mobile client 321 may enumerateall data assets on mobile client 322 taken near the GPS location of oneof his video recordings taken at the event and around the times of theevent. Then mobile client 321 may retrieve all those data assets. Whenthe data assets are transferred, the data assets may be tagged with theEvent ID and associated metadata. Mobile client 322 may similarlyperform such an action to mobile client 321.

After data transfer, either mobile client 321 or 322 or both may opt toterminate data connection 340.

Exemplary Correlation Method

FIG. 4 illustrates an exemplary method for correlating data assetsregardless of platform.

In step 410, an Event ID (also called a Pounce™ ID) may be obtained.This Event ID is provided as the minimal amount of metadata to correlatethe resulting data asset with other data assets. Data assets with thesame Event ID may be retrieved together. A data asset may eventuallyhave more than one Event ID.

In step 420, correlation metadata prior to capturing the data asset maybe applied. Correlation metadata may include, but is not limited to, theEvent ID. Other correlation metadata may include the identity of theperson capturing the data and a location stamp.

In step 430, data capture commences. During capture, correlatingmetadata may be captured along with the media. For example, GPS samplesmay track location during a mobile video recording in progress.

Once recording is complete, at decision block 440 a decision is made ifthe resulting data asset is to be shared in place via a peer-to-peerconnection or uploaded to a web service.

If the data asset is to be uploaded to a web service, a groupcorresponding to the Event ID may have to be joined 450 and thenuploaded to a central store 460 via the web service. Alternatively, ifthe data asset is to be shared in place, there is no need to upload thedata.

In step 470, the data asset may be modified multiple times. In 471,additional metadata, correlating or otherwise may be applied.

In step 473, the data asset may be post-processed. For example, if theuploaded data is a still JPEG, it may be cropped. Another exampleincludes correlating the data asset with other data assets andintegrating them together.

At decision block 472, a decision of whether to share the final dataasset may be made.

If the final data asset is to be shared, in step 480 the data asset maybe stored in a memory different than central storage, such as a memorystick or a DVD-ROM, and the data asset may then be ready fordistribution. Otherwise, the data asset remains in storage for furtherretrieval and other actions. The following discussion describes thisprocess in more detail.

In step 410, the Event ID may be created in many different ways providedthat the Event ID is sufficiently unique to identify one or more sets ofdata assets covering at least a portion of an event. Again, while anevent may correspond to an actual real world event, for purposes of theembodiments described herein, an event may be a set of recordings thatare to be collected together for any reason, real-world, artistic, orotherwise. For clients enabled with consumer software to create an EventID (e.g., Pounce™ software), the Event ID may be a globally uniqueidentifier (“GUID”) generated by software such as guidgen.exe fromMICROSOFT™. Alternatively, the Device ID of the capturing device may beused. On a cell phone if the transport is short message service (“SMS”),the SMS ID may be used as an Event ID. For yet another example, forTwitter™ based distribution, the PID plus additional supporting text maybe used for an Event ID.

In step 420, pre-capture metadata is to be applied to the one or moresets of data assets. The metadata may include the Event ID as well asany metadata globally applicable to the one or more sets of data assetssuch as, but not limited to, name of the person capturing the event,date/time stamp, location, and device type. The pre-capture metadatapreferentially is stored within a data asset persisted as a file. Forexample the MICROSOFT™ advanced systems format (“ASF”) supports theplacing of support metadata within the file format itself.Alternatively, the metadata may be stored separately and distributed asa companion file for the one or more sets of data assets.

In step 430, media is captures, which may include capturing an event andmetadata at the same time. Examples include, but are not limited to,location markers, timestamp, and telemetry. As in step 420, the metadatamay be stored along with the data in the asset as a file, but couldalternatively be stored separately in a companion file.

In scenarios where there are a small number of users with a small numberof files, the files may be retrieved or edited in place in a peer topeer network. In such situations, a web service may not be necessary,and per step 440, may bypass joining a group and participating in a webservice.

However, in peer to peer connections, the larger the number of users,the more cross-connections and communication overhead is required. For Nusers, there are (N*(N+1))/2. Thus, for example, three users wouldrequire six cross-connections and four users would require tencross-connections. Since the overhead to maintain connections wouldbecome prohibitive, for a large number of users, a web service may bepreferable. Per step 440, a non peer-to-peer scenario may involve a webservice by which a group may be joined corresponding to the Event ID andthe data asset being uploaded to a central storage. For example, whenuploading a videofile to a web-based video repository, a user may uploadan associated text field which in turn may contain an Event ID. As partof the upload, the user may also upload license data (not shown). As analternative to a web service, a service on a corporate LAN might beemployed as well.

In step 470, once the data asset has been captured, a decision to modifythe data asset may be made. The modification may include, but not belimited to, one or more of adding metadata, applying post-processingincluding combining data assets via correlation metadata, or persistingthe data asset for distribution in steps 472 and 480.

In step 471, metadata is applied after the data is captured. In fact,the Event ID may be edited or changed, or an additional Event ID mayeven be added. Further, custom metadata may be added; or if customfields of metadata are redefined, old metadata may be made consistentwith a new format. In the case of external data assets being added, asdescribed in reference to FIG. 2, items 230 and 260, metadata may beconverted to match metadata of other data assets. In step 473, dataassets are post-processed. Such post-processing may include editing thedata asset, e.g., cropping photos, changing color distribution;providing other special effects; or combining data assets. For example,combining and correlating an audio track and a video track may beconsidered to be post-processing or perhaps are just combine virtuallyfor during the process. An example is a police video from a car in theform of a file and audio overlaid from the device that recorded audiowhile outside of the car.

In step 472, if the decision is to share the data asset separate fromthe platform as disclosed, the data asset may be stored on a separateportable media such as a memory stick or DVD-ROM. Alternatively it maybe stored on a network drive. The sharing may be in the form of a singlefile, or alternatively in a distribution of one or more data and one orseveral metadata files.

In this exemplary method embodiment, the result is a single multimediafile or single distribution that combines multiple data assets that havebeen correlated for consumption by a user.

Exemplary Hardware Platform

The platform implemented by the various embodiments described herein maybe based on commodity hardware. In the alternative, custom hardware maybe applied to implement the platform for improved speed or efficiency.The server hardware may be based on a standard personal computer (“PC”)architecture, or may be based on cloud computing as will be describedlater. The client may be a PC client with the ability to capture media,or a custom mobile client as described in FIG. 5.

(i) Standard PC Architecture

Both servers and clients for the platform implemented by the variousembodiments described herein may be based on a PC architecture.Specifically, there may be a processing unit comprised of one or morecentral processing units (“CPUs”) that each has one or more cores. Theremay be an on board read only memory (“ROM”), e.g., a basic input/outputsystem (“BIOS”), which manages boot up. For working memory, there may berandom access memory (“RAM”); for storage, including virtual swap space,there may be one or more hard drives. There may further be aninput/output interface to support both serial and parallel operations.The input/output interface may contain support for a mouse and keyboard,or the PC may have a separate interface. All these parts may beconnected on a bus to multiplex data and instruction connectivity. Theremay be a fast bus for CPU communications to RAM and a slow bus forinput/output operations. Connectivity between the two buses may behandled by a northbridge.

The input/output interface may include connectivity to a number ofperipherals. Expansion cards or on-motherboard functionality may bedirectly connected to the bus. Further, there may be a video graphicscard, which, in turn, may provide connectivity to a video monitor. Ifthe PC is to be networked, there may be a modem or network interfacecard (“NIC”). NICs may support a wide range of protocols. The PC mayeven have a cellular modem. Audio functions such as microphone andspeaker support may be supported by an audio card. Interface for opticalstorage such as CD-ROM, DVD-ROM, and BluRay™ disks may be handledthrough the input/output interface. Other portable storage such asmemory sticks/thumb drives and legacy floppy drives may also be handledthrough the input/output interface.

In the case of clients, for media capture, cards supporting fastinput/output such as Universal Serial Bus (“USB”) 2.0 and Institute ofElectrical and Electronics Engineers standard no. 1394 (“IEEE 1394”),also known as “FireWire” interfaces can be supported by a PC, e.g., highresolution microphones and video cameras. However, fast input/output canbe any sort of data acquisition, including, but not limited to, locationsampling, telemetry, or other streaming data feeds.

In the case of servers, large arrays of storage, such as Redundant Arrayof Inexpensive Disks (“RAID arrays”) are common. Multiple CPU andmultiple core configurations with relatively large amounts of RAMprovide support for large numbers of users.

(ii) Exemplary Mobile Client

FIG. 5 shows a client embodied in the form of a multimedia cell phone.Exemplary mobile client 500 contains functionality for cell phonecommunications including, but not limited to, transceiver 520 and a setof coding functions usually embodied in a chip set 530. Known computingfunctions such as a processor, bus, system clock, and memory, includingbut not limited to RAM, ROM, flash ROM and connectivity hardware such asantennae are not shown.

Mobile client 500 is powered by power functions 560. Information isentered via buttons 540, and potentially via a screen 570 if it is atouch screen. Visual output is provided through screen 570. Media can becaptured through media input functions 510, including video and stillcameras 511, and microphones 512. Metadata including correlatingmetadata may be entered via metadata collector functions 550. Thesefunctions are described in detail as follows.

Client 500 may receive a signal via the antenna (not shown), and thensignal may then be sent to the receiving function of the transceiver520. After the transceiver, there may be a function to determine whattype of signal is being received (not shown), to distinguish betweenbackground noise and actual voice input. After the signal type has beendetermined, the signal may be sent to codec 531 in the coding functions530. The decoded signal may then be forwarded to filters 533 and errorcorrector 532, both of which may improve the quality of the decodedsignal. Finally, the signal may be forwarded to an appropriate renderingmechanism. For example, if the signal is a voice signal, then it may besent to a transducer such as a speaker (not shown); if the signal is SMSor web browsing data or other software data, it may be sent to anappropriate software application and then displayed on screen 570.

A signal may be then generated in response, be it through voice from atransducer such as a voice microphone (not shown), through buttons 540,or other inputs such as a stylus (not shown) or screen 570 if itsupports touch screen functions. Regardless if the input is voice orsoftware, or a combination of the two, the signal is then forwarded tothe appropriate codec 530 for coding, and then for transmission in thetransmission function of the transceiver 520, and then to the antenna(not shown).

Client 500 has the ability to capture media. Contemporary cell phonesalso support on board media input 510 for video and still data via acamera 511 and via a microphone 512. While these are known forms ofinput, further media input may be through data connections such as podcasts and other streams (not shown).

Client 500 further includes the ability to respond to triggering events.For example, a Radio Frequency ID (“RFID”) reader (not shown), canprovide a software notification that an RFID card has been read anacknowledged. In turn, the camera may be turned on for recording.Another example is a vehicle collision detector creating a softwareevent to turn on recording.

In additional to capturing media and triggering events, client 500includes metadata collection functions 550. For example, samples oflocation metadata may be collected by location service receiver 551.Geolocation metadata may include Global Positioning System (“GPS”)metadata. However, because GPS is prone to error from GPS signalsbouncing off of buildings in urban settings, geolocation metadata mayalternatively be determined by triangulating signal strength or weaknessfrom different cell towers with known locations. For relatively immobileclients, receiver 551 may collect geolocation metadata via internetprotocol (“IP”) address.

Another form of metadata is date/time data. Obtaining date/time metadatafor client 500 may be accomplished using an onboard clock (not shown).Alternatively, date/time metadata may be obtained from a signal from acell phone tower.

Yet another form of metadata is text, generally entered by a userpushing buttons 540. Client 500 may utilize a software application bywhich the user enters metadata values via the 10 key pads or via a touchscreen 570. Such traditional button entry would be handled via buttonentry function 552.

Alternatively, custom functions for hotkeys 553 may be employed.Specifically, hotkeys may be used to enter common tags. For example, auser may enter the “#” key to indicate the use of a metadata tag key.Further, the user may enter a number, for example “2” if the number wasnot already defined and text in the value for a metatag, such as “Jane.”Thus, the next time the user wanted to tag a photo with the name “Jane”,the user would press “#” to trigger metadata mode and then press “2,”and then “Jane” would then be associated with the photo. If no photo ormedia file was available upon triggering the hotkey, the user may beprompted to delete or edit the hotkey for different use. This would easethe otherwise cumbersome task of associating metadata with a restricteduser interface.

Other metadata input/output functions, may include, but not be limitedto USB and FireWire. Input/output functions on client 500 may includebut are not limited to: (1) providing other sources of media capture,(2) providing sources of detectors of triggering events, and (3)providing sources of metadata capture. Accordingly, in alternative toFIG. 5 where all of these functions are on board the mobile client 500,these functions could be accomplished via peripheral hardware.

Not shown are custom chips to aid in functions typically enabled viasoftware. These include but are not limited to data compression chipsand encryption algorithms.

Power functions 560 provide power to the all of client 500. Typicallythis is in the form of a battery 561. But a charger/AC input 562typically recharges the battery or provides direct power.

An alternative form of charging may be performed using inductance coils.In situations such as law enforcement, remembering to charge a clientdevice may not always be present in a user's mind. By implementing thecharger as an inductance system including a program to manage thecharging. Specifically, the program may determine when the system isbeing charged, how much power to convert, and when the system is to bedisconnected. The on board mobile client processor (not shown) or aseparate processor may alternatively be used, as may the on board mobileclient RAM (not shown) or separate memory be used. The mobile client mayalso have an inductance coil to receive energy, and then to the powerfunctions 560 and there to the battery.

An off board charger may be configured as an inductance coil thatconnects to an electrical power source, and may further include aprocessor and memory to indicate when to charge and how much to chargethe client device. The off board charger may even further include anindicator having a charge status indicator in the form of colored lightemitting diodes (“LEDs”) or, alternatively, an LED array.

Alternative charging configurations may include placing the mobileclient's charger 562 off of the device and elsewhere on the person ofthe user. For example, coils may be placed in the sole of a user's shoe,in or on the seat of the user's pants, in the user's belt, or in the webgear of the user. The corresponding charger may further be disposed onthe gas pedal of the user's car or even on a car seat, which could thencharge the client device whenever the coils are proximate.

(iii) Mobile Client Chassis

In order to support vehicle mounting, the mobile client may be placedwithin a chassis of aluminum or other material, with an extendable armthat connects to the windshield of a car, much like a rear view mirror.The chassis may have tamperproof qualities and may include one or moresecurity locks that prevent theft or unauthorized modification of themobile client.

The mobile client components may be divided into two portions to supporta secure recorder configuration: (1) the data gathering portion,including cameras and microphones, and (2) the storage portion whichcontains the stored data. This latter storage portion may be secured ina hidden and hardened portion of the vehicle to prevent theft orunauthorized modification. The data gathering portion should be inpublic view; therefore, in order to best capture the surrounding events,the ability to conceal this portion is limited. However, if a maliciousactor steals the camera, the actor steals only the device but not thedata, which then may be later used for its intended purpose, or even tohelp determine who stole the camera. The hidden portion may also behardened such that in the event of an accident, the data storage may berecovered or still functional to upload data.

Several alternatives may be exercised to prevent stealing the datagathering portion. One alternative is to integrally mount the datagathering portion into the vehicle's dashboard, such that removalrequires removing the dashboard and unbolting the device. Thisalternative would lengthen the amount of time to remove the datagathering portion to greatly increase the chances that a would-be thiefwould be caught. For scenarios where an officer wants an option to movethe camera, the camera may be removed from the vehicle when not inoperation, and stored securely in a separate location. In eitheralternative, a would-be thief is deterred or prevented from stealing thedata gathering portion.

Connectivity

Embodiments of the platform support data connectivity. Data connectivityfrom a PC architecture client is primarily from a network interfacecard, for example an Ethernet card, or in the alternate from a dial upmodem. Data connectivity from a mobile client most commonly would be viaa cellular connection. Clients are not limited to just one form ofconnectivity, and may have multiple data connections. For example, a PCclient may have both a modem and a network interface card; or a mobileclient may have both a cellular and Wi-Fi connection.

Connectivity support is not limited to data connectivity. Connectivitysupport may also be for voice data as with ordinary cellularconnectivity. Support for device communication, e.g., Bluetooth support,may also be available.

Various embodiments of the client may support a full network stack. Atthe data link layer, client support may include, but is not limited to,Ethernet support for PC clients and Wi-Fi support for both PC clientsand mobile clients. For Network/Session/Transport layer protocols,support may include transmission control protocol/internet protocol(“TCP/IP”), user datagram protocol (“UDP”), and other protocols. Forapplication layer protocols, file transfer protocol (“FTP”) support foruploading large media files may be available, hypertext transferprotocol (“HTTP”) may be available for web access, and simple mailtransfer protocol (“SMTP”) may be available for email access.

Embodiments of the platform may support peer-to-peer connectivity, bywhich client devices may create an ad hoc network to access and tradefiles. In the alternative, embodiments of the platform as disclosed mayalso support dedicated networks. For example one remote client mayvideotape an event and another client may record the audio of an event.The remote clients may support infrared data association (“IRDA”)standards and may be able to transfer files to each other. Since IRDA isslower than Wi-Fi, the remote clients may support Wi-Fi and set up aprivate ad hoc network between the two. Finally the remote clients mayparticipate in a dedicated network along with PC clients.

Extended Platform

Since multimedia files are large, even with compression, embodiments ofthe platform may create large amounts of data. Accordingly the data maybe stored on cloud computing centers. By integrating with cloudcomputing, the embodiments described herein may make available largeamounts of storage and further provide access to compute-intensiveapplications available on cloud computing platforms. However,integration may result in degraded performance and over-reliance on athird party.

There are a number of compute-intensive applications that may be hostedalone on large computer clusters. Face recognition is one suchapplication. However, when such applications and databases are hosted ona cloud computing node, in addition to having higher availability of alarger number of compute resources, the application is not inherentlynetworked. One application of the various embodiments described hereinincludes a client camera capturing a video frame or still image,extracting out a subject's face using standard algorithms, and thencalling a cloud computing database.

FIG. 6 indicates an exemplary implementation of such a scheme. Cloud 610has a number of web servers, hosted applications, including but notlimited to the face recognition software, and databases, including butnot limited to a database of faces. An application host 630 may includean uploading web service but may delegate cloud computing requests toapplications from non-cloud clients 641 and 642.

An embodiment of the uploading implementation may be described withrespect to remote client 641, which has a still camera, and PC client642, which has a number of still photos stored as files. PC client 642with may store a number of still photos with the name of the personstored with the file as correlation metadata in the form of a tag. ThePC client 642 may upload the photos to the application host 630 via aweb service. The application host may then store the photos to centralstorage in cloud 610 where is the photos may be available for retrievalvia search and correlation. In the alternative, PC client 642 may uploadthe untagged photos and then tag them with metadata after the uploading.

An embodiment of the download process may be described with respect toremote client 641, which takes real-time still photos at an event. Thecontext may be in law enforcement surveillance or in photojournalism.Regardless, the user of remote client 641 may automatically tag thephoto with the names of individuals on the photos. Remote client 641 mayupload the photos to application host 630 via a web service which maythen store the photos on a central store in cloud 610. Once on the webservice, the remote client 641 may request the photos to beautomatically tagged. The application host may then invoke a facerecognition application running on the cloud to retrieve all photos thatare similar to the faces in the photo uploaded from remote client 641.Where the retrieved photos do not have sufficient similarity to thephoto uploaded from remote client 641, third party databases may beinvoked. Based on the tags provided from PC client 642 as well as allother clients that stored photos in the database, the relevant tags maythen be applied to the photo uploaded by remote client 641.

Because the correlation and presentation are integrated together FIG. 2,item 220, custom features such as autotagging are available. Moreimportantly, because the embodiments as disclosed integrates withexternal data FIG. 2, item 230, and provides for metadata correlationFIG. 2, item 260, it can integrate with third party databases such asfacial databases; including integration with cloud computingapplications.

This integration with cloud computing applications provides for fasterresponses. Accordingly, remote client 241 might receive the names of theindividuals just taken in a photo immediately. Additionally, the remoteclient 241 might receive additional information such as company name orother professional information. On a variation of the above scenario,the user could have taken a picture of a product in a store, used objectrecognition software in the cloud, and similarly retrieved productinformation. The foregoing is exemplary and not intended to be alimiting or exhaustive list of possible applications of extending thepresent platform to include cloud computing.

The integration of the described herein may not only provide feedbackregarding the accuracy of the facial recognition, but may also work toimprove matches. For example, if clients 641 and 642 are organized intoa group, the likelihood of retrieving false matches from a generaldatabase is removed by limiting the face recognition database only tomembers of the group. Furthermore, by adding autotagged photos to thegeneral database, especially after post capture corrections, provides anever improving sample of photos to determine facial matches. Facialrecognition capabilities may be augmented by taking changes over time,such as hair style, and the growth of mustaches and beards, and couldreturn name information but also time period information.

One problem with cloud computing integration is that data may be widelydistributed geographically, thus creating wide variances in networklatency and performance. For example, in FIG. 6, cloud 610 may belocated in or near Seattle, Wash. but cloud 620 may be located in ornear Washington, D.C. Thus, a user located in Seattle would likelyexperience slow data retrieval if the data were based in cloud 620 butbetter performance of data based in cloud 610.

Accordingly the application host 630 may manage offline, edge, andnearline data by (1) caching data on application host 630 itself and (2)invoking server affinity, which guarantees that a particular server, orat least a particular cloud, is to serve data to a particular user. Datathat is known to be needed commonly may be placed on the applicationhost that is nearline. Data that is known to be needed by usersgeographically local to the cloud, but is not commonly used may bepinned to a local server or a local cloud via server affinity. Placingdata redundantly on different edge points of the cloud may not be costor storage prohibitive because cloud computing provides large amounts ofstorage. Further, data that is not time sensitive may be stored offlineor arbitrarily on the cloud.

Another problem with cloud computing is over-reliance on a single cloudcomputing provider. For example, where the central storage is hosted ona cloud for a production system such as for law enforcement, cloudfailure means data storage is lost. If the law enforcement force was ina major city such as New York with 1,000 policemen on duty, down time ofa cloud for 1 hour would be a loss of almost a man-year of data. Cloudcomputing is a relatively new technology, cloud computing brownouts andblackouts are possible. Additionally, much data, in particular in lawenforcement scenarios must be made secure.

The application host 630 may integrate storage across clouds 610 and 620from different cloud computing providers and mirror. Alternatively, theapplication host 630 may implement a RAID scheme, which may subdividedata across three clouds, all from different providers. Security, whichmay include auto-encryption, may be enhanced since no single cloudprovider is likely to have all the data. In both cases, storageavailability is improved.

Exemplary Client Software Platform

FIG. 7 illustrates an exemplary client software platform 700. The clientsoftware platform is based on an ordinary cell phone software stack.However, to support the custom applications, at each stage, acorresponding custom layer may be added. The main hardware interface maybe the device drivers 710 and 720. Standard drivers 710 providefunctionality to stock hardware. However, custom drivers 720 may berequired for custom hardware. Typically drivers will communicate withthe cell phone operating system 740. Examples include SYMBIAN™ andANDROID™, the operating system for the Open Handset Alliance. Theoperating system 740, including the operating system kernel may requireextensions 750 to provide support for custom software events, as anexample. To provide the necessary information to the operating system740 and the extensions 750, software “shims” 730A and 730B may interceptand hook notifications from the drivers and may provide alternativefunctionality. Standards libraries 760 may be built upon the operatingsystem.

For custom functionality, the exemplary platform may include customlibraries 770, mostly exposing programmatic support for the softwareeventing model 751, and to custom hardware as exposed by the customdrivers 720. Finally, applications 780 may be built on top of thelibraries 770. The following discussion will cover each of these areasin more detail.

Standard drivers 710 may include drivers for stock cell phone hardwareincluding, but not limited to, buttons 711, screen 712, and memory 713.There may be other stock hardware, including e.g., a GPS receiver (notshown).

Custom drivers 720 may include drivers to support non-stock hardware fora cell phone. Custom drivers may be included in various embodiments bywhich a mobile client comprises a cell phone with additional hardwarefunctionality.

One example of a custom driver 720 is a custom network stack 721 tocompensate for the presence of a partial network stack, as in variouscell phone embodiments. However, full implementations are to supportfunctions typically not on a cell phone including Wi-Fi connectivity,FTP, to name a few.

Other examples of custom driver 730 include a custom USB implementation722 and a custom Plug 'n Play (“PnP”) implementation 723. Not all cellphones support PnP, which is the automatic installation of hardwaredrivers and automatic configuration and provisioning thereof. However,some cell phones may have additional hardware added for telemetry andmetadata purposes. A full USB stack 722 and support for PnP 723 mayprovide such functionality in various embodiments. As USB is not theonly serial interface stack, it optionally may be replaced with someother serial data interchange stack.

Standard drivers 710 and custom drivers 720 may serve to forwardcommunications to the operating system 740. Generally there will be anevent queue and a series of modifiable event handlers (not shown).Modification of the event handlers may include a recompilation of theoperating system 740. An alternative is to create extensions 750 to theoperating system 740 to isolate any necessary custom functionality. Thisincludes a custom software eventing model 751.

Software events are different from the physical events that may becaptured via media and multimedia, in accordance with the variousembodiments described herein. Other terms for software events include“triggers” and “notifications.” A software event may include a messagethat software sends when some occurrence discernable by software occurs.An example includes a hardware button being pushed, a driver triggeringa software event that sends a message to all applications subscribing tothe event for which a button has been pushed. If an application has anevent handler, that event handler will contain functionality as to whatthe application should do when a button is pushed.

A software eventing model may support software events for customhardware. For example, if a custom driver provides an interface for anRFID trigger, the custom driver may send a notification to the operatingsystem extension's internal event queue. The internal event queue maythen forward the notifications through custom libraries 770 toapplication 780. The application 780 may then handle the event of theRFID trigger by turning storing the camera's preview buffer and startingthe record function of the camera. Alternatively, software events may beprocessed via a modification of the operating system's 740 event queue.Additionally, event notifications from applications 780 may bedisseminated via the software eventing system 751.

The standard drivers 710 preferably are not modified, or are modified aslittle as possible and custom drivers 720 should be isolated from theoperating system as much as possible. To enable custom eventing, in somecases, the communications may be intercepted or hooked by module 730Afor the standard drivers and 730B for the custom drivers. For example,if a particular button sequence is to bypass the operating system andperform a custom function such as a triggering a hotkey mode, it may benecessary to intercept the button pushes and redirect execution to ahotkey executable. Further by way of example, if a combination of thedefault operation and a custom option is required, the button pushes canbe simply hooked thus passing through the push notifications to theoperating system at the same time triggering a custom button pushsoftware event handler.

The operating system 740 and the operating system extensions 750generally will expose their functionality via an application programminginterface (API). Standard libraries 760 generally provide function callsor object models to aid programming applications. Standard libraries 760generally are distributed along with the operating system. Libraryextensions, that is custom libraries 770 provide function calls orobjects models to support custom hardware, operating system extensions750, or to provide new additional functionality on top of the originaloperating system 740.

There is a wide range of applications 780 now enabled by this extendedplatform as will be described as follows.

(i) Patch Manager

Patch manager 781 is an application that handles updates for the driversand installed software. There are two ways of patching. The firstoptimizes saving memory, the second optimizes versioning. For bothversions, the flash memory contains a lookup table mapping locations inmemory for functions. When an application calls a function, it goes to alookup table that was populated by the table mapping from flash. In thisway, the function can proceed to the location of the function andproperly execute.

The first example of patching stores the lookup table values in a wellknown location. It then stores drivers, the operating system,applications, and any other binaries in other locations of the memory.Each binary is allocated more memory than it currently needs in order toaccount for future patches. When the patch manager receives notice,which includes all new binaries and a new lookup table, perhaps over thenetwork, or via an SMS message, the patch manager triggers a softwareevent. The software event shuts down all software except for thosenecessary for patch functions. The patch manager first overwrites thelookup table, and then overwrites all binaries with the new binaries.The patch manager then forces a reboot. The reboot then restarts themobile client, populates the software lookup table with the lookup tablein flash. When an application calls a function, it then will go to thenew location and to the new binary.

While the first example conserves memory, it does not provide provisionfor falling back in the event bad patches are installed or patches wereimproperly installed. In a second example of a patch manager, the sametriggers and events are used as in the first example. However,additionally, the location of the current lookup table is stored. At awell known location, several bytes of empty storage store each newlocation of lookup table. The original lookup table address is in thefirst location. The lookup table of the first patch is in the secondlocation, and so on. Whenever a patch is installed, lookup table and allbinaries are stored contiguously. Upon reboot, the operating systemlooks for the last lookup table and then installs the lookup table intosoftware from that location. In the event a patch has to be backed out,the patch manager can decrement the list of addresses of lookup tableversions, reload the older lookup table, and in doing so revert back tothe older software.

(ii) File Manager

File manager 782 is an application that provides advanced filemanagement for on board removable storage. Typically when removablestorage such as memory sticks are in a cell phone, the assumption isthat the user will remove the memory stick and place it in a PC toremove and otherwise manage files. However, for scenarios such assecurity cameras where removing the memory stick may take long periodsof time, an ability to manage files as not to run out of memory isrequired. To address these requirements, the file manager 782 containsfour main functions: (1) deleting files upon upload, (2) round robindeletion, (3) metadata tagging upon upload or download, and (4)enumeration of data assets.

For the first function, as files are written to media, eventually anotification, perhaps from SMS will trigger uploading all files notcurrently open. Where a file is open, optionally the trigger will forcea closing of a file and a reopening, for example with the onboard videocamera is currently recording.

For the second function, where uploading is sporadic or non-existent,the file manager may maintain a queue of files and implement a firststored first out memory management approach. Specifically, when memoryis no longer available, the oldest files will be deleted and that spaceallocated for a new file. The file manager may implement packing andcompression algorithms as well.

For the third function, upon upload or download the file manager checksthe Event ID of the data asset. If it is null, it populates the metadatafield with the current Event ID on the mobile client. Where the Event IDis associated with other metadata, for example the name of the event,that field is populated if null. In the alternative, it may preventupload or download.

For the fourth function, the file manager provides not only forenumeration of data assets, but simple search via metadata. The filemanager maintains a table of data assets on the machine and maintains alist of metadata fields and offsets. When request, for example from SMS,or alternatively via a network requires a list of data assets matchingmetadata criteria, it sequentially iterates through all the data assetsin the data asset table and retrieves the file name if the data assetmatches. It then writes the file name into a text file. The text file isthen returned over the data connection. The text file may be formattedwith XML or with a proprietary format to aid enumeration andpresentation on the receiver of the file.

(iii) Quality of Service (“QoS”) Failover

A QoS Failover application 783 manages multiple media inputs, forexample a video camera and a microphone. Consider the case where themobile client is in a police car and the police officer is wearing awireless video camera and a wireless microphone configured for theplatform as disclosed. Further consider the case such that the videowireless connection has a lower range than the microphone wirelessconnection. While the police officer is near the car, both video andmicrophone are streaming data. When the officer exceeds the distance forthe video connection, but not the microphone, the QoS Failoverapplication can cut-out storage only for microphone. When the officerthen exceeds the distance for microphone, the QoS Failover applicationcan then place a marker in the file indicating no receipt. When thepolice officer returns to the car, the microphone and video canrecommence recording according to range.

(iv) Client Synchronization (“Client Sync”)

Consider the case where data assets from multiple mobile clients are tobe correlated. The Client Sync application provides the client sidesupport to aid consistency and reliability of media and metadata acrossthe clients, hence the term sync. Upon activation, the Client Syncregisters the mobile client with a web service or equivalent. The webservice provides an Event ID and in turn the mobile client via theClient Sync uploads data assets to the web service. The data assets aretagged with the Event ID. The Client Sync also provides location samplesto the web service, as to allow for tracking of the mobile client. Whendata assets are uploaded, the Client Sync also appends to the data asseta checksum. Upon upload, the web service can validate the authenticityof the uploaded data asset. The checksum (also called a heartbeat) canalso contain additional metadata. For its part, the web service can alsoprovide additional metadata. For example, consider a law enforcementscenario with multiple officers. Each of the officers has a mobileclient and already is associated with an Event ID. When a particularincident occurs, the web service can provide an Incident ID that getsassociated with the officers proximate to the event. This Incident IDgets associated with all data assets uploaded until the web servicerescinds the Incident ID. In this way, later the data assets associatedwith the Incident ID may be easily identified.

Identifying relevant mobile clients to send the Incident ID relies onknowing the location of the mobile clients. If an officer indicates thatan incident has occurred, either through a hotkey or a message, the webservice attempts to identify which other mobile clients with the sameEvent ID are proximate. Usually this may be done via GPS. However, asdescribed in the hardware section above, GPS is prone to error from GPSsignals bouncing off of buildings, location may alternatively bedetermined by triangulating signal strength or weakness from differentcell towers with known locations. For relatively immobile clients, suchas surveillance tower cameras, geolocation via IP address may beemployed.

(v) Remote Control

The remote control application 785 takes advantage of the full networkstack on the mobile client. Consider the case where the mobile clienthas an IP address and is fully participating on a network. The remotecontrol application contains a proxy to intercept remote instructions,either through SMS or from a direct IP connection to a remote site.Accordingly, a remote user can fully control the client, either formaintenance such as patching and upload of data assets.

(vi) Preview

The preview application 786 is not an application to preview data.Rather it is an application to guarantee that media captured in thepreview buffer of a video camera is recorded along with the rest of theevent. On video cameras with preview, a memory buffer stores the firstfew seconds of data. This buffer could store an arbitrary amount oftime. Upon recording, the video begins recording at the time oftriggering, in other words at the start of the event to be recorded.However, for security cameras and other applications, it may bepreferable to store the previous few seconds in the preview buffer aswell to ensure complete recording. Accordingly, a RFID triggeredsecurity camera function in a mobile client configured with the platformas disclosed would store not only the event upon triggering the camera,but also the first 10 seconds. In this way, the video would store notonly the event, but the precursor events that led to the event.

(vii) Custom Charger

Mobile devices such as the mobile client as disclosed require constantrecharging. A custom charger application 787 determines how much chargeis left in the battery, and provides alerts when charge is low.Furthermore, in cases of inductance charging, the custom charger canstore information about how much power is needed to recharge thebattery. It could also store rules on detecting when charging coils weresufficient proximate and stable to trigger charging functions.

(viii) SMS Based Mobile Client Control

The ability to receive short message system (SMS) messages and triggersoftware events provides a general mechanism to remotely control aphone. Specifically, a mobile client that has an SMS capable transceivermay monitor SMS calls and perform software routines in responses.Specifically, one of the custom drivers 725 could be an SMS driver toallow the ability to monitor SMS calls for custom commands. Calls to andfrom the SMS driver could be intercepted and hooked. When a predefinedSMS message to a predefined location is received, it could trigger anevent in event model 751 which in turn could be utilized in anapplication 788.

An alternative embodiment is to have a daemon application 788 withoutthe driver or event model to directly monitor SMS messages such thatwhen a predefined SMS message to a predefined location is received itwould perform a particular prespecified task.

An example application for monitoring SMS messages relates to lawenforcement. A “stealth” recording feature may be implemented where amobile client receives a SMS message which includes a phone number, theSMS message instructs the mobile client to start recording itsenvironment and transmit the audio and/or video over the received phonenumber. Another SMS message could instruct the mobile client to stoprecording and transmitting. Variations on this scenario where an SMSmessage instructs the mobile client to initiate a phone call but doesnot record, as to the type of media to transmit, and whether to use thecalling number as the destination to transmit recorded media will beappreciated by one of ordinary skill in the art.

In particular, Table A as follows provides an exemplary mapping ofevents to SMS Messages to be interpreted a control instructionsoperative to control a mobile client's stealth recording function:

Event SMS Message Description Recording Start #*77 Start recording mediaCall Start #*27 + phone # Initiate call to phone #. If phone # is notspecified, then initiate call on the sender's phone #. Recording/CallStart #*7727 + phone # Initiate call to phone # and start mediarecording. If If phone # is not specified, then initiate call on thesender's phone #. Recording Halt #*74 Terminate ongoing media recording.Call Halt #*24 Terminate ongoing phone call. Recording/Call Halt #*7424Terminate the ongoing media recording and phone call.

Table A. Event to SMS Mapping for Stealth Feature

The mobile client may optionally send a response SMS message to thesender to confirm receipt of messages, proper operation of the client,or error messages. The SMS messages could be used as a protocol,including but not limited to triggering software events by the receiver.For example, an SMS acknowledgement for starting a call could trigger asoftware server to start receiving streaming media. Another examplewould be to receive an error SMS stating that a connection could not beestablished, and displaying the error message as part of a computerapplication.

The mobile client could receive SMS messages from any SMS source. Theseinclude but are not limited to other mobile devices and computerapplications with a cell that could transmit SMS. Accordingly, protocolscould be established over SMS between two mobile devices or with acustom computer application with capable of transmitting and receivingmessages compliant with the SMS protocol.

Although the SMS messages above are merely exemplary, a sample sessionmay be as follows. The mobile client receives an SMS message #*271-555-555-1234. In response, the mobile client starts a phone call with1-555-555-1234, and immediately sends an SMS acknowledgement. The mobileclient then receives an SMS message of #*77. The mobile client immediatestarts recording video and audio. If the recording cannot be performed,the mobile client would send back an SMS error message to1-555-555-1234. Periodically, the mobile client will upload files to anFTP server or other host. Alternatively, the calling number1-555-555-1234 could provide a connection to an FTP server orequivalent. In another alternative, the server to which 1-555-555-1234provides connectivity could trigger a streaming receipt upon receivingthe acknowledgement SMS from the mobile client. The mobile client, uponreceiving #*74 and send an acknowledgement SMS and would halt recording.A new recording could be established the mobile client receiving #*77and again halted by the mobile client receiving #*74. Similarly uponreceiving #*24, the mobile client would stop the call and send anacknowledgement.

(ix) Other

The above applications are not intended to be an exhaustive list ofapplications available. General utilities such as profile management(not shown) are available. A profile manager could store userinformation such as name to be used as default metadata. It could alsostore results about memory usage, bit rate, and resolution of storedmedia.

Commercial utilities can also be implemented on this platform. Forexample in the case of law enforcement, the mobile client might have ahot key for a police officer to indicate the officer has been attackedor shot. The video would trigger to store the preview buffer as well asthe current events, and a central station could call for backup askingother police to converge on the attacked officer's GPS location.

Another law enforcement example is the alternative log on/passwordutility for undercover policemen. The alternative log on/passwordutility accepts two passwords. One password activates all the lawenforcement functionality on the mobile client and displays a userinterface specific to the law enforcement functionality. A secondpassword simply displays an ordinary user interface expected on anordinary cell phone, and additionally may activate a call for backup tothe central station. Specifically, when an undercover policeman beginsto be suspected to be a cop by those he or she is investigating, thepoliceman might be searched. Ordinarily, if the policeman is searched,and the mobile client is found, an inspection of the mobile client mightarouse suspicions that the undercover policeman is a cop. The undercoverpoliceman could enter a the second password into the alternative logon/password utility while would show the ordinary cell phone userinterface and may help convince others that the undercover policeman isnot a cop. In the meantime, the backup call to the central station wouldallow other policemen nearby to rescue the undercover cop.

Commercial utilities are not limited to law enforcement. The mobileclient may support applications not specific to its video or networkingcapabilities. An example of a commercial utility not specific to lawenforcement is where the client could store an audit function (notshown) that indicates that an unauthorized person has attempted to openthe tamperproof chassis. An example of using the SMS based mobile clientcontrol feature, such as used by the Stealth feature described above, isthe case of a lost mobile client. The mobile client may host anapplication that upon receiving a message, such as an SMS text message,may un-mute a mobile client with its ringer turned off or alternativelymay activate a mobile client that is in sleep mode, and then ring thedevice. In this way, a mobile client that has been misplaced in a room,where the ringer has been turned off or cannot otherwise be locatedsimply by dialing the mobile client's number.

Exemplary Non-Client Software Applications

Once the data assets are tagged and available on a central store, theymay be searched, correlated, tagged, edited, and presented eithercollectively via a web service, or to a closed set of users via astandalone application. The platform as disclosed gives rise to bothtypes of applications.

Regarding a web service, groups may correspond to a group. The datareviewed by the groups, the content of the tags, and comments, made bythe group may be queried for possible advertising. Because the group isself selecting, the fit of ad selection will improve a selectionalgorithm, based strictly on scanning media content, and metadata. InFIG. 2, the external data store 230 as provided with correlatingmetadata in 260 could be the correlation of advertising.

Network Operations Center

FIGS. 8A and 8B show a non-client based commercial application of thepresent platform, a network operations center (“NOC”) of a lawenforcement agency.

The NOC provides the capability of viewing individual videos. In videofeed 810, a video, or potentially another form of multimedia will berendered. Annotation tools 811 provide allow a user select one or moreframes of running video, superimpose text, links, or other content,perhaps associating to a single object, such that the annotations renderupon playback. Many tools presently exist for annotation of video andmultimedia in general. Running commentary box 820 provides a controlwith which to view commentary and commentary edit box 830 provides aplace to add additional commentary. Commentary text is persisted to adatabase and is associated with the multimedia file being rendered.Commentary is distinct from annotation in that the text is visible atall times in the running commentary box 820 during rendering, whereasannotation only appears for specific frames and is superimposed overvideo as rendered in the video feed 810. Associated videos, stills, textor other multimedia may render at the same time in browser 840. Forexample, links from the annotation may render in browser 840. Map 850may also display the location of the video. For example, if the video isby a police car in hot pursuit, the map may show a point renderinggeolocation of the path of the pursuit itself.

The NOC has a “supervisor mode” shown in FIG. 8A, where data feeds,including video and audio, are coming from police officers proximate toan incident. Specifically, a police officer encounters an incident andpresses a hot key telling the NOC that an incident is under way. The NOCcan start an incident record and the police officer's feed appears inCar Feed 1, item 851. Because it is the first feed, it also appears inthe main video feed 860. The map 880 then centers on the location ofthat particular officer. The incident ID may be used as an Event ID.

On the map, the user can view other officers near to the incident andcan either manually select other officers individually by clicking onthe officers, or by dragging and dropping a bounding rectangle over thelocation. In the alternative, the application can use a predefineddistance and automatically join other officers. Each joined officer'sfeed then shows in feeds 861, 862, 863, 864, 865 and so on.

When an officer is joined, the officers' feed is associated with theEvent ID and defines a group. Thus joining officers within a predefineddistance and unjoining officers that go out of that predefined distanceis an autogrouping feature.

The main video feed 860 may display which of the feeds 861, 862, 863,864 or 865 to concentrate on. Alternatively, the main video feed mayautomatically switch between which officer is closest to the incident.Note that the video feed is not a composite view, but rather is anexample of multiple videos being shown side by side based on correlationmetadata.

If the data is being viewed after the fact, browser 870 can bring updata from the law enforcement's case database. All media files with theEvent ID may then be automatically tagged with the Case ID from the casedatabase. In this way, data from the case database may be provided in anintegrated view along with the media.

Law enforcement mobile clients have redundant local storage that iseventually uploaded for archiving. The application can provide a scannerfunction (not shown) that can manage uploading from the mobile clients,can tag the file as archived thus preventing redundant storage. In thealternative the scanner function could seek similar clips based on EventID and location and timestamp information and delete redundant clips.Because of the redundant storage, the NOC can provide securityfunctions. Specifically the scanner function could also detect filestampered with by noting files with the matching data with differentchecksums. Additionally the NOC can check checksums on uploaded files todetect files intended to spoof the NOC.

Vertical Applications

The embodiments as disclosed may be used for both personal andcommercial vertical applications. Advantages include but are not limitedto: (1) reducing the price of camera monitoring via use of commoditizedever miniaturizing hardware and ever cheaper storage, (2) guaranteedfull capture of an event by implementing preview buffers, (3) customeventing where a camera may be triggered on an arbitrary event, (4)integration with other media via correlating metadata, and (5)integration with third party data via correlating metadata. Thefollowing sections will describe an exemplary application for personaluse, an exemplary application for law enforcement, an exemplaryapplication for security cameras, and will more generally enumerateother exemplary commercial scenarios.

Mobile NOC

The NOC application may be accessible via a web page on a notebookcomputer or other web enabled mobile device. Accordingly, all orportions of the NOC application may be viewed while in the field by alaw enforcement officer.

An example application would be allowing a junior officer to patrol anarea and to view indicia of incidents that occurred within a time framein the present locale. In this way, the officer could more quickly learnthe crime history of his location even though he had never been in thearea before.

One embodiment would be to let an officer travel through a neighborhood.The officer would enter a time range of data to browse for the area, forexample over the past year. The NOC application would pick upgeolocation information of the roaming officer such as GPS location anddisplay a map in the NOC would show pushpins indicating incidents withinthe specified time frame. The pushpins could be color coded, or havesize or shape changes indicating the severity or recentness of theincidents. The officer could click on the event and view the video orassociated media to learn more. In this way, the officer couldfamiliarize himself as to the crime history of an area in near-realtime.

The mobile NOC need not be in real-time. An officer could be in an area,and if he wished to learn more about the crime history of his location,he could open the NOC application and specify his current location andsearch for crimes within a time area and various search criteria. Inthis way, upon entering a crime scene the officer might find relatedcrimes not readily discernable from the crime scene itself. For exampleif there was an assault at a crime scene, the officer could quicklyidentify other assaults nearby and consider suspects from those crimesas well as leads from the present crime scene.

Peer to Peer Scenario Embodiment

Video cameras, digital cameras, and digital recorders are to beubiquitous among law enforcement officers. It is not uncommon forseveral policemen, each with their own camera or recorder, to beproximate to a crime in progress. The embodiments as disclosed enablesharing and the automatic correlating of the resulting data assets.

FIG. 9 illustrates a scenario where three police officers, one with apersonal camera 923, the other two with cell phones enabled per thedisclosure 921, 922, are proximate to the same crime scene 910 inprogress and start taking photos. The still shots capture differentparts of the crime scene 910, some overlapping, some not. Mobile client921 captures still shots 931, mobile client 922 captures still shots932, and digital camera 923 captures still shots 933.

Because mobile clients 921 and 922 are enabled per the disclosure, theyare able to perform a peer to peer connection. Per the embodiment asdisclosed, the initiating client 921 identifies its device id, andenters a tag called “Robbery 1” and sends both to 922 to use as an EventID and as extra metadata. Mobile client 922 sends an acknowledgementback to 921, thus fully establishing data link 940. Mobile client 921then downloads via IRDA all photos time stamped during a time framecorresponding to that of the crime from 922, and mobile client similarlydownloads all photos form 921. Both clients automatically tag the photosmoved through the data link 940.

Standard camera 923 is not configured per the present disclosure andcannot perform peer to peer transfer as mobile clients 921 and 922.Instead, the users of mobile clients 921 and 922 establish a group (notshown) on web service 960 using mobile client 921's Device ID as anEvent ID. The user of standard camera 923 then joins the group and whenstandard camera 923 uploads chosen photos to the web service over datalink 953, the photos are automatically tagged with the Event ID and themeta tag “Robbery 1.”

A user, such as a police supervisor back at headquarters, who is amember of the group (not shown) subsequently may access the web service960 via a PC 970 over a data link 980. The user may further uploadrelated photos, such as getaway scenes of the robbery crime scene 910 ofpersons of interest driving away, and again via the data link the photosare automatically tagged with the Event ID and the meta tag “Robbery 1.”The user may also query, edit, tag, and combine photos on the PC.

It is to be emphasized that in the above scenario, media is not limitedto photos. Video or audio could have been uploaded as well. The webservice could have been a standalone application running on a network.Most importantly, the event, here exemplified by a crime scene 910, neednot have been at the same time or place. An Event ID could tie togetherany media regardless if they previously were related at all.

Law Enforcement Scenario Architecture Embodiment

FIG. 10 illustrates an exemplary law enforcement architectureembodiment. Consider the case where multiple active duty police officersare carrying mobile clients per the embodiments as disclosed. Policeofficers on foot carry a mobile client 1010, and police cruisers carry amobile client 1030 in a chassis where the video camera is connected tothe windshield, and the data is stored in a tamperproof, securelocation. Exemplary tamperproof, secure locations include the trunk ofthe vehicle or glove box of the vehicle. In particularly hot climates,the trunk or glove box could be become so hot as to be beyond theoperating temperature range of the data storage hardware portion may bein a secure cage integrated with the chassis of the vehicle, in a shadedand ventilated portion of the backseat. Alternatively, the tamperproofsecure location could be integrated with cooling, such as an insulatedcavity with refrigeration powered by electricity from the vehicle. Host1020 provides data asset storage and archiving and backup services.Supervisor application 1040 is based on Wi-Fi connectivity and is hostedon a PC in a NOC. Both mobile clients 1010 and 1030 and the chassis of1030 are based on the exemplary client described in FIG. 5 andsupporting text. The supervisor application is as described in FIG. 8Band supporting text.

Host 1020 receives data assets from mobile clients 1010 and 1030 both insynchronous and via asynchronous upload. A web service 1021 can remotelyconfigure a mobile client 1010 by checking the existing profile 1011sending an updated profile via client application 1012. In general,client application 1012 functionality includes providing networkconnectivity to the host. The profile includes and is not limited tofrequency of receiving an IP signal to poll the status of the mobileclient. This signal is also called a heartbeat. Upon configuration, webservice 1021 will send out a heartbeat according to that frequency, forexample every 15 seconds. Client application 1012 will receive theheartbeat and return geolocation information, such as GPS coordinates.In the event of connectivity error, website 1027 would be able toindicate which mobile clients 1010 and 1030 have failed to connect orhave not transmitted a heartbeat.

Because a mobile client 1010 and 1030 in the field broadcast geolocationinformation, a network operating center (“NOC”) can get mobile locationinformation from the host and know in real time the location of mobileunits. This enables map applications. For example a NOC application maydisplay selected units on a computerized view of a map, perhapsaccessible via a web site 1027. Alternatively, particular mobile clientsmay be searched by geolocation. By way of another example, the NOC canhave the location of a crime scene and the current location of a mobileunit 1010 and automatically map the optimal route for the mobile unit toget to the crime scene. This feature is also known as a “Get Me There”feature.

As client application 1012 captures media, it stores the correspondingdata assets on storage 1013. Via synchronizing application 1014, themobile client 1010 may either push the stored media via file transferprotocol (“FTP”) to the file transfer server 1022, or the synchronizingapplication 1014 may be remotely triggered, for example via an SMScommand, to affirmatively upload the stored media as data assets.

Once the data assets are uploaded to file transfer server 1022, acrawler application 1023 periodically scans the uploaded data assets.The data assets are placed into a folder queue and optionally associatedwith metadata, for example in the form of metadata tags corresponding toincident, or case. Transcoder 1024 converts the file formats of the dataassets into a standard format, for example one based on H.264. Oncetranscoding is completed, if the transcoding is successful, crawlerapplication 1023 archives the original file and stores the transcodedvideo to store 1026 and a reference to the transcoded video to database1025. In the event of error, the uploaded data asset is placed intoanother directory for later review.

One error that may occur in particular with data assets stored as H.264compliant files is the loss of the file index. A file might be partiallyuploaded, but because the file index is stored at the end of the file,none of the partial uploaded file is usable. Prior to transmission,synchronization application 1011 might store the H.264 index in aseparate file, and redundantly upload the index. Because the index ismuch smaller than the media data in the H.264 file, corruption is lesslikely. In the event the uploaded H.264 file is truncated, the completeindex, uploaded separately may be used to decode the portion of theH.264 file that arrived. If the H.264 file arrived intact, then theredundant index may be discarded.

Once data assets have been stored in the database 1025 and store 1026,they are accessible via a website 1027. The website may includefunctionality to search and view videos including external data such ascase data. For example, it may enable “computer aided dispatch”, wherean officer is dispatched to a crime scene, incident information isentered by the dispatch via website 1027 and uploaded to database 1025,data assets are uploaded from mobile client 1010 or 1030, tagged withmetadata corresponding to the incident, and the searched and viewed incomposite afterwards.

Website 1027 may also provide various ways to quickly review videofiles. For example website 1027 might provide thumbnail or videocapviews of videos to aid visual search. In some cases, a scroll bar mightbe displayed to allow the user to scan through individual frames in thevideo, rather than merely viewing a default frame.

Website 1027 may also integrate with external websites. For example, asupervisor may access website 1027 and upload a video to a public socialnetworking site as part of a public alert such as missing child alert(“Amber alert”).

Website 1027 may also provide visibility to monitor the status ofuploaded data assets, and provide for bandwidth management andprioritization. For example, website 1027 may show what data assets havebeen uploaded, transcoded, or what percentage of the process iscomplete. It may also show which data assets have errors or areotherwise corrupt. Specifically, the website 1027, may review thecontents of the file transfer server 1022 directories which storeuploaded data assets, archived data assets, and corrupt data assets. Itmay further receive notifications from crawler application 1023 as towhat percentage of an operation such as transcoding is complete. If adata file is to be prioritized for upload or transcoding, the website1027 might provide a control to configure the crawler application 1023accordingly.

Mobile client 1030 shows an alternative architecture for a client. Herecar service 1031 combines the functionality of client application 1012and synchronizing application 1014. Additionally, it provides networkconnectivity to a supervisor application 1040. Supervisor applicationprovides the ability to provide remote control over client 1030 and toview configuration, status, and stored data assets on client 1030. Onmobile client 1030, the mobile configuration and the captured dataassets are stored in the same store 1032. Data assets are captured andreviewed locally by application 1033.

The following describes features and advantages enabled by the platformas disclosed for law enforcement.

(i) Chain of Custody

Police and security video may be used in evidence at trial. However, inorder to be admissible, the evidence must be stored such that anunbroken chain of custody can be made, ensuring that the evidence wasnot tampered with. With the embodiments as disclosed, a metadata tag tostore an Officer ID and a metadata tag to store a timestamp and actionmay be made. When the police video is stored, the tag stored the OfficerID of the owner of the mobile client. When the officer uploads forstorage, the officer authorizes the upload which again tags the policevideo with Officer ID, timestamp, and action. When the data is receivedand archived, the archival officer may again tag the police video withhis Officer ID, timestamp and action. Thus at all stages, the chain ofcustody of the video may be verified. For further security the metadatamay be encrypted. Alternatively, checksums may be archived in a databaseto detect later tampering.

(ii) Providing Video ID

Because all video is uploaded and archived, citizens who have beenstopped may request an identifier for the video clip capturing theirstop and a limited access account. The citizen may then go to the lawenforcement agency's web site, log on, and then view the footage oftheir stop. This will provide an incentive for officers to avoidexcessive force and can provide evidence for and against anyculpability.

(iii) Monitored Interrogation

When suspects in custody are interrogated by police officers, typicallythere is a camera to memorialize the event. However, a determined policeofficer can disable a typical security camera. If a mobile clientenabled with RFID and preview is in the room, it will be difficult totamper with the camera. Specifically, an officer with an RFID badgeenters the interrogation room. The RFID badge not only triggers videorecording, it also makes use of preview mode to capture the first 10seconds prior to entry. Therefore, even if the officer were to tamperwith the camera, the camera would have captured the first few secondsand would detect the tampering.

(iv) Emergency Dispatch

Often officers enter hazardous situations. When an officer is assaulted,the officer may hit a hotkey on the mobile device that acts as a panicbutton. The mobile client sends a notification that includes the officeridentity and the officer's location which then goes to the NOC. Thesupervisor mode of the NOC includes map showing nearby officers whichare also broadcasting their locations via GPS or other locationservices. The officer at the NOC may then quickly dispatch the proximateofficers to the location.

(v) Blackout Versions of Video

Police videos are often used as evidence in trial. The embodiments asdisclosed support post process editing of videos. Because it may bedesirable to black out portions of video as unfairly prejudicial, bothdistrict attorneys and defense lawyers may edit out or black outportions of the video to be left out of the trial record. One way to dothis is via professional post processing tools. A less expensivealternative would be to use an overly tool (a block of non color overtop of the video) to place overlays blocks over the portions of thevideo to be blocked out. In this way, not only could portions of framesbe provided rather than full blackout, faster turnaround of editingcould occur.

Security Camera Scenario Embodiment

A variation of mobile officers carrying mobile clients is to have a meshof security towers covering a geographic area. This would enable closecircuit surveillance over an arbitrary area with minimal installationcosts. Examples of locations where these may be applicable include butare not limited to be college campuses, airports, prisons and high crimeareas. It is to be noted that a mesh of camera towers need not belimited to law enforcement scenarios. For example, a mesh of cameratowers at a sporting event or an entertainment event would provideimproved media coverage of the event. Each of the towers would have amobile client configured with a full network stack to allow remotecontrol over IP and Wi-Fi and a file manager to manage file upload andfile deletion. Both the network stack and the file manager are discussedin FIG. 7 and the supporting text.

Each camera is network enabled and is independent in that it is a selfcontained recording and storage device that adds metadata such as timeand location. Because the cameras house these on board capabilities, allthat is needed locally is power. No communication cabling would berequired.

Each camera could be configured to host a synchronizing application thatwas able to detect proximate cameras. An embodiment would be to refer toa web service that tracked the location of each installed camera, andfrom knowledge of its local location, could select proximate cameras.

To aid in backwards compatibility, where a camera did not have thenecessary hardware initially, a hardware block with the missinghardware, e.g. to enable network connectivity, could be attached, andthe necessary software components installed. Accordingly, previouslypurchased hardware could be made to be compatible for mesh applications.

However, collectively the cameras act together as a mesh of cameras thatgive a law enforcement agency a comprehensive view of what happened atthe same location or close by locations at the same time. A NOC'ssupervisor mode could quickly search for all uploaded data assets fromthe camera at a particular time and location, and then view and editthem together.

Note that the cameras all networked peer to peer devices. Accordingly,an officer cold come to a location where there were several devicespresent, such as the scene of a crime or other event, and coulddetermine on his laptop or cell phone the locations of all the cameras.The officer could then choose cameras to download video to his laptop orcell phone, or by default download all videos from the surroundingcameras.

A camera need not start out as part of a mesh. A camera may operateindependently until another camera is added to the network to establisha mesh. Thus a single camera, perhaps mounted on a pole, could providead hoc surveillance in a relatively short period of time. As time wenton, other cameras could be incrementally added to establish a mesh.Cameras could also be removed as necessary.

Sports Replay Scenario Embodiment

A non-law enforcement scenario for using mesh is to replay sports games.Professional and other sports team's videotape games and practices tobetter assess points of weakness and to identify ways to improve theteam and individual player's performance. An ability to view a play oraction from multiple angles would provide a comprehensive way to assessthe action.

By providing a mesh of multiple cameras incorporating the disclosuresherein, multiple cameras may synchronize to the same clock. If foursurrounding cameras were configured to view a play and weresynchronized, the configuration would provide a quad view.Alternatively, cameras in the mesh could provide a mix of differentviews, for example a coach's view, views from two different players, andperhaps two sideline shots.

Per editing software using the metadata features disclosed herein,metadata specific to game analysis indicating for example the particularplay, the context of the game and players involved could be added. Forexample for a football game, metadata indicating that a particularsegment of media represented a first down, a running play, where Joneswas the runner could be added. In addition to providing context uponplayback, it also provides metadata enabling searching and filtering onkinds of plays, players, and other elements of interest in analyzing thegame.

The game analysis metadata need not be rendered, but could also be usedfor statistical analysis of the game. One could determine likelihood ofhow a team or player reacted according to different situations. Forexample in football, one could determine that a team was significantlymore likely to do a long pass on a first down than other teams.

Cameras using the present embodiment have a sharable format, thusdifferent teams could agree to share media with each other. Because thecamera's clocks could be synced to a global source, e.g. a webaccessible value of global time, the different media could besynchronized with each other, even though they had been recordedseparately by different parties. Accordingly, different views designedfrom different perspectives may be correlated together.

Other Possible Commercial Embodiments

Commercial embodiments are not limited to law enforcement. The followingare some other commercial scenarios supported by the platform.

(i) Taxicab Scenario

The embodiments as disclosed can replace a taxicab's meter box andcamera with a single device. Currently, most taxicabs install a meterbox to show the fare charge. Some taxicabs also install a camera to showthe face of the driver to the passenger. However meter boxes only showcharges, not distance, route, or other information that may be ofinterest to the passenger.

A taxicab hosted client as disclosed, with a video camera and a GPS orother geolocation tracker, integrated with a map database and anexternal charge database may perform the roles of the meter box andcamera for a lower price and with greater functionality including butnot limited to: (a) a view for the passenger with start and end pointson the map, with the specified route as calculated by software alsoshown so the passenger could proactively request a different route; (b)the passenger view displaying progress on the route by showing thetaxicab's GPS tracking samples as the trip progresses; (c) the passengerview additionally showing not just taxicab fare information, but alsotime and distance information, traffic alerts, predictions of arrivaltime; (d) a log of the trip could be stored and uploaded via the mobileconnection, or in the alternative could update via Wi-Fi upon parking,in order to provide an audit trail to a central office; (e) thepassenger view could provide average trip times and other aggregatedinformation from the audit information; (f) the client integrated with aprinter could give a receipt with the information from the view uponrequest of the passenger; and (g) uploaded audit information could beused in training new taxicab drivers.

(ii) School Bus Scenario—Child Tracking

The present embodiments could monitor pickup and drop off of children.Currently typical school buses do not have onboard cameras. Accordingly,determining whether a child was dropped off at the correct location onthe way home, or in the alternative determining if a child was correctlypicked up relies on the bus driver's memory. In the case of a missingchild, the camera could provide objective evidence as to whether thechild was on the bus or not.

A school bus hosted client of the present embodiments with an eventingsystem for door opening and closing, a video camera with preview, and ageolocation tracker, integrated with a map database includes but is notlimited to the following advantages: (a) when the school bus door opensor closes, the custom eventing system could trigger recording of thevideo camera; (b) the video camera would have several seconds of previewbuffer, so would record a those several seconds along with the actualevent of a child entering or exiting the school bus; (c) the customeventing system could trigger turning off the video camera; (d) thegeolocation tracker could trigger upon turning on the camera as well andalong with a map database indicate where the pickup or drop offoccurred; (e) the information could be archived to a central office viacellular upload or Wi-Fi; and (f) the archived information could beaggregated for statistics and in where no errors of pickup or drop offoccurred, the information could be deleted to save storage.

(iii) Trucker Scenario

The present embodiments could monitor accidents for bulk transportationsuch as semi-rig trucks. Currently semi-rigs do not have onboardcameras. Accordingly, disproving the liability of a semi-rig in theevent of an accident is difficult.

A truck hosted client of the present embodiments with an eventing systemtriggered by a vehicle crash detector such as disclosed in U.S. Pat. No.4,161,228 which triggers airbags, a video camera with preview, ageolocation tracker integrated with a map database includes but is notlimited to the following advantages: (a) when the vehicle crash detectordetects a crash, the custom eventing system could trigger recording ofthe video camera; (b) the video camera would have several seconds ofpreview buffer, so would record a those several seconds along with theactual event of the crash; and (c) the geolocation tracker could triggerupon turning on the camera as well and along with a map databaseindicate where the accident occurred and upon upload could notify acentral office as to dispatch a response team. Additional advantages maybe realized by further integration with a telematics system whichprovides additional digital telemetry about the state of the truck. Forareas that did not have cell phone coverage, alternative data transportsuch as packet radio could be implemented.

(iv) School Bus Scenario—Accident Tracking

As a hybrid of the previous School Bus and Trucking scenarios, anembodiment of the present disclosure could monitor accidents on schoolbuses as well. Currently school buses do not have onboard cameras.Accordingly, determining causality and harm to children on the bus isdifficult to prove.

A set of school bus hosted clients as described by the presentdisclosure, specifically with one or more cameras recording eventsoutside of the bus and one or more cameras recording events within thebus, with an eventing system for crash detection, a video camera withpreview, and a geolocation tracker, integrated with a map databaseincludes but is not limited to the following advantages: (a)comprehensive monitoring for how events outside of the bus impact eventsinside the bus, including but not limited to bus collisions; (b) thevideo camera would trigger from a collision and would have severalseconds of preview buffer enabling it to record several seconds prior tothe collision; (c) the crash detector event could also trigger acollision message to a central location along with geolocation dataindicating where the collision occurred; (d) the information could bearchived to a central office via cellular upload or Wi-Fi; and (e) thecentral office could collect media from cameras not on the bus toprovide different angles of the accident.

The descriptions of the above scenarios are not intended to be anexhaustive list of uses of the present embodiments and are only some ofthe many possible applications made possible.

CONCLUSION

In compliance with the statute, the subject matter of this applicationhas been described in a language more or less specific as to structureand method features. It is to be understood, however, that theembodiments are not limited to the specific features described, sincethe disclosure herein comprise exemplary forms of putting the presentembodiments into effect. The present embodiments are, therefore, claimedin any of its forms or modifications within the proper scope of theappended claims appropriately interpreted in accordance with thedoctrine of equivalents and other applicable judicial doctrines.

What is claimed is:
 1. One or more computer-readable media of a mobileclient storing computer-executable instructions that upon executioncause one or more processors to perform acts comprising: interpreting aninstruction into at least a command, the instruction being sent from acomputing device remote to the mobile client and received by a hardwareinterface of the mobile client; retrieving a software eventcorresponding to the command according to a predefined command to eventmapping; and executing an event handler that handles the software eventresponsive to the command, wherein the event handler initiates a mediacapture device of the mobile client to capture a data asset of a crimescene and further associates the data asset with a case identifier forthe crime scene.
 2. The one or more computer-readable media of claim 1,further comprising: interpreting an additional instruction from a serverand received at the hardware interface of the mobile client into anupload command; and uploading the data asset as captured by the mediacapture device to the server in response to the upload command.
 3. Theone or more computer-readable media of claim 1, wherein the instructionis from a server, further comprising sending an acknowledgment messageof the instruction to the server that causes the server to startreceiving the data asset.
 4. The one or more computer-readable media ofclaim 1, wherein the instruction includes an identifier of a recipientdevice that initiated the instruction, and wherein the mobile client iscommanded by the instruction to transmit the data asset to the recipientdevice based on the identifier.
 5. The one or more computer-readablemedia of claim 4, wherein the identifier of the recipient device is aphone number.
 6. The one or more computer-readable media of claim 1,wherein the instruction is a short message service (SMS) messagereceived from an additional client device, the SMS message beingreceived by a text message capable transceiver of the mobile client. 7.The one or more computer-readable media of claim 1, wherein the mediacapture device is a camera or a microphone, and wherein the data assetis a video file or an audio file.
 8. A computer-implemented method,comprising: sending, from one or more computing devices, an instructionto a transceiver of a mobile client, the instruction to command themobile client to use a media capture device of the mobile client tocapture a data asset; receiving, at the one or more computing devices,an acknowledgement message from the mobile client in response to themobile client acting on the instruction, the acknowledgement messageinitiating a streaming reception of the data asset from the mobileclient; and integrating, at the one or more computing devices, the dataasset that is received via the streaming reception with an additionaldata asset that is captured by another mobile client to form a compositedata asset.
 9. The computer-implemented method of claim 8, furthercomprising receiving an error message from the mobile client anddisplaying the error message via a computer application in response tothe mobile client being unable to carry out the instruction to capturethe data asset via the media capture device.
 10. Thecomputer-implemented method of claim 9, wherein the instruction includesan identifier of a recipient device that initiated the instruction, andwherein the receiving includes receiving the error message following adelivery of the error message to the recipient device based on theidentifier.
 11. The computer-implemented method of claim 8, furthercomprising sending a halt instruction that stops the capture of the dataasset following the receiving of the acknowledgment message.
 12. Thecomputer-implemented method of claim 8, wherein the sending includessending the instruction as a text message, as radio-frequencyidentification (RFID) data, or as an instruction initiated by an eventdetector.
 13. The computer-implemented method of claim 12, wherein theevent detector is a vehicle crash detector and the instruction isinitiated by the vehicle crash detector in response to a detection of avehicle crash event by the vehicle crash detector.
 14. Thecomputer-implemented method of claim 8, wherein the instruction is ashort message service (SMS) message, the SMS message being received by atext message capable transceiver of the mobile client.
 15. Thecomputer-implemented method of claim 8, wherein the media capture deviceis a camera or a microphone, and wherein the data asset is a video fileor an audio file.
 16. A server, comprising: one or more processors; andmemory having instructions stored therein, the instructions, whenexecuted by the one or more processors, cause the one or more processorsto perform acts comprising: sending an instruction to a transceiver of amobile client, the instruction commanding the mobile client to use amedia capture device of the mobile client to capture a data asset;sending an additional instruction to the transceiver of the mobileclient, the additional instruction commanding the mobile client toupload the data asset; receiving the data asset as captured by the mediacapture device in response to the mobile client receiving the additionalinstruction; and correlating the data asset with an additional dataasset captured at a crime scene such that the data asset and theadditional data asset correspond to a single case identifier from a casedatabase.
 17. The server of claim 16, wherein the acts further comprisereceiving an acknowledgement message from the mobile client in responseto the mobile client acting on the instruction to capture the dataasset.
 18. The server of claim 17, wherein the acts further comprisesending a further instruction that stops the capture of the data assetfollowing the receiving of the acknowledgment message.
 19. The server ofclaim 16, wherein the instruction is a short message service (SMS)message, the SMS message being received by a text message capabletransceiver of the mobile client.
 20. The server of claim 16, whereinthe media capture device is a camera or a microphone, and wherein thedata asset is a video file or an audio file.